A Generalized Reinforcement-Learning Model: Convergence and Applications

Reinforcement learning is the process by which an autonomous agent uses its experi­ ence interacting with an environment to im­ prove it::; beha.vior. The rVlarkov dec.ision pro­ cess (:\'lDP) modd is a popular way of for­ malizing the reinforcement-learning problem, but it it) by no meant; LIte only \-vay. In Lhitl paper, we show hmy IllaIlY of the important theoretical results concerning reinforcement learning in MDPs extend to a generalized MDP model that includes MDPS, two-player games and .\lUPs under a worst-case optimality cri­ terion as special cases. The basis of this ex­ tension is a stochastic-approximation theo­ rem that redllces fLsynchronolls convergence to synchrOflOlJS cOlnrergence.